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Abstract 

The article explores the challenges of assessment in open distance learning (ODL). The authors argue that 
ultimately assessment should be about improving the quality of teaching and effective learning. It should be 
based on making expectations explicit and public, setting appropriate criteria and high standards for learning 
quality, systematically gathering, analyzing and interpreting evidence to determine how well performance 
matches expectations and standards, and using the resultant information to document, explain, and improve 
performance. However, getting all these variables to work in ODL presents mammoth challenges. How can 
ODL lecturers validate and authenticate students’ written work? How can they tell whether the students’ 
submitted work sufficiently reflects their knowledge and understanding? South Africa has inherited an unequal, 
racially skewed and inequitable educational provision from its apartheid past. This poses serious challenge 
for assessing quality. The article therefore seeks to understand these context-specific challenges of ODL 
assessment at UNISA. 
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Introduction 

We teach in the Bachelor of Education (BEd undergraduate), Postgraduate Certificate in Education 
(PGCE) and Honours Bachelor of Education (Honours BEd) programmes at the University of South 
Africa (UNISA). Running through all these programmes there is a course— Theoretical Frameworks 
in Education —that introduces and consolidates the students’ grasp of philosophical frameworks and 
how they shape educational practice. We will come back to this course in more detail later. 

UNISA (2012) is the biggest open distance learning (ODL) institution in Africa. Unlike campus- 
based, full-time contact institutions that cater mainly for young school leavers entering university 
for the first time, ODL institutions provide access to higher education to mature working students. 
Such students would not be able to obtain a university qualification were they to have campus-based 
full-time contact institutions as their only avenue for accessing higher education. 

Most ODL institutions market themselves to prospective students as open, accessible, flexible, 
supportive and affordable. Presumed in this market discourse is the view that adult working students 
take responsibility for their learning; they learn alone or in small groups; they learn at their own 
pace and in their own time; they learn from a variety of learning materials, including the use of 
audio-visual media; they are active rather than passive learners; they need less frequent help from 
their teachers; they therefore learn from other people besides their teachers, and that they will do 
a lot of self-assessment (Rowntree, 1996). UNISA (2008, p. 2) commits to adhering to “responsible 
open admission policy.” Its open learning approach “gives students flexibility and choice over what, 
when, where, at what pace and how they learn. Open learning is all encompassing and includes 
distance education, resource-based learning, correspondence learning, flexi-study and self-paced 
study.” UNISA’s commitment above resonates with access requirements of the Open University 
(OU) in the United Kingdom (UK), which are based on the principle of “no entry requirements.” Our 
understanding is that nearly all OUUK courses have no entry requirements. 
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On the surface these pronouncements sound noble. However, at a deeper level they pose 
profound challenges with respect to ways of assessing and assuring the quality of ODL teaching 
and learning. For instance, it might not be easy to validate or authenticate ODL students’ written 
work and to ascertain whether the work they have submitted is theirs and that it constitutes a true 
reflection of their level of content knowledge and understanding of the subject matter. While this 
might be reasonably monitored at campus-based full-time institutions, the challenges are more 
pronounced in ODL institutions where students are not required to attend classes in person. In this 
article, which builds on our previous work on the challenges of teaching and learning in ODL 
(Letseka & Pitsoe, 2012, 2013), we explore the challenges of assessment in ODL, with a focus on 
UNISA. The article has six sections. First, we sketch various conceptions of assessment with a view 
to ascertaining how they pertain to the course we offer. Second, we outline formative and summa- 
tive assessment. Third, we describe the course Theoretical Frameworks in Education, its purpose, 
and exit level learning outcomes. Fourth, we briefly sketch ways in which the course is assessed. 
Fifth, we reflect on the challenges of assessment with respect to assuring the quality of teaching 
and learning in ODL. In the final section we provide some concluding remarks. We now turn to 
conceptions of assessment. 

Conceptions of assessment 

There is agreement among advocates of assessment that effective assessment should be ongoing 
and focused on improving students’ learning (Angelo, 1991, 1994, 1995, 1999; Bell & Cowie, 2001; 
Cohen, 2008; Hunsaker, 2004; Shepard, 2000). For instance, Thomas Angelo (1999) is concerned 
that most assessment efforts have resulted in little learning improvement because they are imple¬ 
mented without a clear vision of what “higher” or “deeper” learning is and without an understanding 
of how assessment can promote such learning. He attributes this to “piecemeal” attempts that stem 
partly from a mechanistic, “additive” model of assessment. He suggests that a “transformative” 
assessment-as-culture-change model should replace this. Angelo (1999) argues that “transforma¬ 
tive” assessment-as-culture-change rests on the following four pillars: 

• Building shared trust by lowering social and interpersonal barriers to change; 

• Building shared motivation by collectively identifying goals worth working toward and problems 
worth solving, and considering the likely costs and benefits; 

• Building a shared language by developing a collective understanding of new concepts (mental 
models) needed for transformation; 

• Building shared guidelines by developing a short list of research-based guidelines for using 
assessment to promote learning. 

It follows that if lecturers were to plan their assessment as if learning matters most—and not just 
student learning, but theirs as well—the distance between means and ends would be reduced and 
chances of success increased. Angelo (1995) conceives of assessment as an ongoing process 
aimed at understanding and improving student learning. He argues that assessment should involve 
making expectations explicit and public; setting appropriate criteria and high standards for learning 
quality; systematically gathering, analysing and interpreting evidence to determine how well perfor¬ 
mance matches expectations and standards; and using the resulting information to document, 
explain, and improve performance. Angelo’s views above are echoed by Brown and Knight (2004), 
who argue that assessment techniques work better where learning outcomes have been articulated 
in advance, shared with the students, and assessment criteria agreed upon. Thus assessment helps 
create a shared academic culture dedicated to assuring and improving the quality of teaching and 
learning in higher education. 
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Taylor (2005) advocates the following nine principles of good practice in assessment. For him 
assessment of student learning 

• Should begin with educational values; 

• Is most effective when it reflects an understanding of learning as multidimensional, integrated, 
and revealed in performance over time; 

• Works best when its programs have clear, explicitly stated purposes; 

• Should pay attention to outcomes and equally, to the experiences that lead to those outcomes; 

• Works best when it is ongoing, not episodic; 

• Should foster wider improvement when representatives from across the educational com¬ 
munity are involved; 

• Makes a difference when it begins with issues of use and illuminates questions that people 
really care about; 

• Is most likely to lead to improvement when it is part of a larger set of conditions that promote 
change; 

• Should enable educators to meet their responsibilities to students and to the public. That is, 
it should foster accountability. 

Hunsaker (2004) reiterates the view on assessment as an on-going process oriented around 
curricula change aimed at improving learning. He advocates scholarly assessment, which he defines 
as a tool for helping the lecturers understand the teaching and learning process, a means by which 
lecturers can learn what is working, and where they might do well to approach things differently. 
In this regard assessment should be all about teaching and learning. One of the courses which 
we teach at UNISA, and which we describe below, targets ODL undergraduate and entry-level 
postgraduate students. While its goal is to initiate the students in the knowledge of the theories and 
practice of education 1 , at the heart of its assessment plan is a concern with the best practices that 
ensure students’ learning is improved regardless of the ODL nature of the institution. As Brown and 
Knight (2004) note, assessment should be used to shape and direct future learning. 

Angelo (1991) highlights this purpose more succinctly in his suggestion that assessment should 
provide faculty and students with information and insights needed to improve teaching effectiveness 
and learning quality. The focus should be more on finding out what and how much (or how little) 
students have learned between points A and Z, that is, to establish accountability and improve 
efficiency (Angelo, 1994). In support of this view Cohen (2008) argues that assessment must be 
interactive and proactive. It must be a form of participation. Cohen (2008) regards participation as 
a way of enhancing learning and a form of feedback on the nature of the learning that is, or is not 
taking place. The lecturer and the students should collaborate actively to produce the best perfor¬ 
mance (Yorke, 2003). Cohen (2008) contends that because there is the content that the lecturers 
want the students to understand, discussing readings in class should be central to teaching in that 
it gives the lecturers the opportunity to take a closer look at student learning and to assess how 
well they are learning. 

Participation provides an indication of students’ understanding of their readings. How lecturers 
respond to students’ comments, the feedback they give on various assignments, and the way they 
use the results from class participation could significantly impact student learning. These views 
resonate with the South African Qualifications Authority’s (SAQA) (2001) policy position on assess¬ 
ment. SAQA (2001) posits that assessment in education and training should involve gathering 
evidence of learners’ work so that judgements can be made about their achievements or 
non-achievements. Thus university students should ideally be more involved in their assessment 
on the grounds that they are the relevant stakeholders. However, this is easier said than done. Most 
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forms of assessments are still top-down and largely driven by the educators. Below we briefly 
explore formative and summative assessment. 


Formative and summative assessment 
Formative assessment 

Formative assessment is of critical importance to student learning and retention (Yorke, 2001). Its 
central purpose is to contribute to student learning through the provision of information about 
performance (Yorke, 2003). Formative assessment has the advantage in that it is dialogic. The 
students receive regular feedback on their performance from their teachers. The exchange between 
the teachers and the students is (ideally) mutually hermeneutic in that each is seeking to interpret 
and understand the communications of the other with the aim that the students will become better 
equipped to deal with the challenges of future study. Mentkowski (2000) argues that formative 
assessment should serve as supportive criticism, and an important support for learning and motiva¬ 
tion. In this regard feedback procedures should assist students in forming accurate perceptions of 
their abilities and in establishing internal standards with which to evaluate their own work. For Bell 
and Cowie (2001), formative assessment must be ongoing, dynamic and progressive, informal, 
interactive, unplanned as well as planned. It must be reactive as well as proactive, with the class, 
group, or individual. 

Black and William (1998) are persuaded by formative assessment’s effectiveness in promoting 
student learning across a wide range of educational settings (disciplinary areas, types of outcomes, 
levels). They argue that an important determinant of the effectiveness of formative assessment is 
the quality of the feedback received by learners. In framing the feedback to students, lecturers 
should keep in mind several important and delicate considerations that are neither widely known 
nor understood. Whether lecturers across the different education sub-disciplines recognise this 
critical role of quality feedback to students remains to be seen and is an area that still needs further 
research. 

Summative assessment 

In summative assessment, assessors look for evidence of achievement at exit level. This involves 
identifying some data as relevant to specified goals and the criteria that derive from them (Knight, 
2002). Judgments are made about the match between evidence and criteria. But as Brown and 
Knight (2004) point out, summative assessment has always been a vexed business. First, because 
higher education institutions are generally expected to have learning goals that are far more exten¬ 
sive and complex than mastery of subject matter alone, and they are being held to account for 
student achievement in terms of those goals. Second, a greater range of assessment techniques 
has come into currency, which has introduced substantial practical and theoretical problems, with 
the comparability and aggregation of performances judged by different assessment methods. Third, 
public sector services are nowadays marked by low-trust management systems, when once there 
would have been a greater readiness to trust that good people engaged in worthwhile activities 
would learn the sorts of things that were intended. Thus assessment is supposed to supply evidence 
to bridge the trust gap. Fourth, the eternal concern with value for money has taken a rationalist turn, 
with the belief that it is prudent to specify objectives, measure inputs, assess performance in terms 
of those objectives, allocate the next round of resources to efficient providers and apply sanctions 
to the less efficient. How do these conceptions of assessment play themselves out at UNISA? We 
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mentioned above that running through the Bachelor of Education (BEd undergraduate), Postgradu¬ 
ate Certificate in Education (PGCE) and Honours Bachelor of Education (Honours BEd) programs 
is the course, Theoretical Frameworks in Education. In the next section we briefly sketch the course, 
highlighting its internal structure, and the kind of competencies it seeks to develop. 

The course: Theoretical Frameworks in Education 

The above-mentioned course is one of the central components of the Bachelor of Education (BEd 
undergraduate), Postgraduate Certificate in Education (PGCE) and Honours Bachelor of Education 
(Honours BEd) programs at UNISA. The course explores different ways of thinking about education 
and undertaking research in education. It aims to equip pre-service and in-service teachers 
with the skills to identify, articulate and critically engage with their own underlying assumptions 
about the nature of education, their work in education, and the ideas of others (Venter, Higgs, Jee- 
vanatham, Letseka & Mays, 2006). The course encourages the students to “return again and again 
to the course material, re-examine their assumptions, the questions they are asking, and the answers 
they are discovering.” But by phrasing the challenges of the course as they do, the course design¬ 
ers seem to assume that all the students have been introduced to some prior learning in the 
elements of philosophic inquiry, which is often not the case. 

Theoretical Frameworks in Education is conceptual, philosophical, and contemplative. It is struc¬ 
tured around nine (9) theoretical frameworks or meta-theories in philosophy of education, namely, 
logical empiricism, hermeneutics, systems theory, feminism, phenomenology, critical theory, African 
philosophy, critical rationalism, and post-modernism. The structure of the course is consistent with 
the broader aim of the Honours BEd programme, which is the development of applied competence. 
In this regard, applied competence is understood to involve the integration of the following four other 
kinds of competences: 

• Ability to conduct independent inquiry in a specialised field of education, training or develop¬ 
ment, and to report findings in academically appropriate ways; 

• Demonstrate acquisition of a sound knowledge base and critical understanding of education 
in general and of the chosen area of specialisation in particular; 

• Ability to critically analyse and evaluate knowledge in the chosen area of specialisation, and 
to contribute to systematic and disciplined thinking about educational matters and issues; 

• Exhibit the potential to act as academic leader and expert in the field of education, training 
and development. 

Thus understood applied competence can be briefly summed up as involving the integration of 
foundational competence, that is, the ability to demonstrate understanding of key concepts and 
issues; practical competence, that is, the ability to use what has been learned in some practical 
way; and reflexive competence, that is, the ability to evaluate one’s work and that of others in order 
to identify areas that need to be improved (UNISA, 2010). 

The course’s reading material comprises a study guide, about which the students are informed 
“works like a teacher.” It will “structure your learning, explain concepts and direct you to other parts 
of the module at appropriate times. It will facilitate your learning through the development of a 
dialogue involving guided activities and feedback” (Venter et al., 2006, p. vi). Then there is the 
prescribed textbook, which “is like a resource person to whom you go for advice. The textbook 
explores some of the main schools of thought influencing education decision making and research.” 
Third are Tutorial Letters. These are a series of communiques that provide students with useful 
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information on the course: “tutorial letters are the means by which we [lecturers] maintain an ongo¬ 
ing relationship with you [the student],” the study guide declares. Tutorial Letters provide information 
on the purpose, nature, structure and learning outcomes of the course. Finally, the students are 
encouraged to keep a workbook or file or computer folder. This is a record of the students’ thinking 
and development throughout the programme. The students are encouraged to use a workbook, file 
or computer folder to record their responses to activities, write notes and summaries, and generally 
record their ideas as they work through the programme. “Thoughts are a bit like fish,” the students 
are warned. “They can easily slip away,” which is why it is important to make a habit of writing ideas 
down as they occur, and to revisit and reflect on them. 

The students are encouraged to read through the prescribed book; develop a broad overview of 
the theoretical frameworks, be able to compare the main ideas of the various theoreticians, and to 
contrast similarities and distinguishing characteristics. In the end, they should be able to draw on 
one or more theoretical frameworks to resolve issues pertaining to specific educational practices. 
For instance, the students might be required to draw on a particular theoretical framework to iden¬ 
tify curricular subjects in the teaching/learning context, and to develop appropriate instructional 
strategies to critically reflect on their practice as teachers. We now turn to the course’s assessment. 

How Theoretical Frameworks in Education is assessed 

So far we have argued that assessment should be an ongoing process and a means for lecturers 
to ascertain what is working, and what needs to be done to improve student learning. We have 
established that the main purpose of assessment is to provide lecturers and students with informa¬ 
tion and insights in order to improve teaching effectiveness and learning quality. Moreover, we have 
highlighted that assessment should be a collaborative endeavour between the lecturers and the 
students. It follows from the above exposition that assessment is a powerful tool for use by lectur¬ 
ers to make timely and effective curricular interventions that will ensure effective students’ learning. 

We now briefly outline the assessment plan of the course, Theoretical Frameworks in Education. 
We should mention that the planning and preparation of assessment should involve four critical 
aspects: first, why is assessment being conducted? Second, what competences should be assessed? 
Third, who should be involved in assessment (Teachers, lecturers, students)? And fourth, how 
assessment should be carried out (methods, instruments, resources and procedures)? 

It should be clear by now that Theoretical Frameworks in Education is not about rote learning 
and/or regurgitation of memorised information. Instead the course requires students to critically 
engage with what they have learned, and to “try things out and evaluate the outcomes” (UNISA, 
2010). The course is assessed through a combination of essay assignments and an examination. 
The students are required to submit three to six essay-type conceptual, philosophical, and contem¬ 
plative assignments and obtain an aggregate of at least 50% towards their continuous, formative 
aspect of the assessment of the programme. The students are cautioned that the following verbs 
will be used in the assignment and examination questions: “analyse,” “compare,” “define,” “describe,” 
“discuss,” “evaluate,” “explain,” “illustrate,” “interpret,” “juxtapose,” “outline,” “synthesise,” and “tab¬ 
ulate,” to mention a few. This caution is intended to alert the students to the conceptual, philo¬ 
sophical, and contemplative nature of the course. How lecturers are able to instil among ODL 
students these conceptual, philosophical, and contemplative inclinations is a challenge to which we 
now turn and reflect upon. 

The challenges of assessment in ODL 

We mentioned above that ODL institutions market themselves as open, accessible, flexible, sup¬ 
portive and affordable. Indeed most ODL institutions operate “no entry requirements” and/or other 
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use-friendly alternative admission criteria such as the Senate Discretionary Admission (SDA) require¬ 
ments and/or recognition of prior learning (RPL). The reason for this is that most ODL institutions 
provide access to higher education to mature working students who would otherwise not be able 
to obtain a higher education qualification were they to have campus-based contact higher education 
institutions as their only option for accessing higher education. And yet noble as this provision of 
access might seem, in the end the principle of “no entry requirements” raises a number of red flags 
with regard to assuring the quality of teaching and learning in distance learning. 

What we want to do in this penultimate section is to highlight some of the contestations around 
the notion of “quality” in education with a view to showing how such contestations have implications 
not only on how we assess, but also on how we assure the quality of academic work submitted by 
students in ODL. The notion of quality in education has always been a contested terrain (Ball, 1985; 
Frazer, 1994; Green, 1994; Massy, 2003). For instance, in the mid-1980s when Christopher Ball 
(1985) asked: “What the hell is quality?” he was grappling with the muddle that surrounds the mean¬ 
ing of the term. In her book —What is Quality in Higher Education —Diana Green (1994, pp. 12-13) 
argues that quality, like “freedom” or “justice,” is an elusive concept. In the same vein Malcolm 
Frazer (1994, p. 103), one of the contributing authors in Diana Green’s book above, acknowledges 
that quality in higher education is a complex idea about which there is no agreement either between, 
or within countries about what is meant by quality. But Frazer offers what seems to us a useful 
understanding of the notion of quality. He suggests that quality should be about what students 
have learned—what they know, what they can do, and what their attitudes are, as a result of their 
interactions with their teachers, department and university. 

There is agreement among commentators on quality that quality is a measurement of some kind. 
For instance, while Frazer (1994) suggests that quality should be about what students have learned, 
Green (1994, p. 5) contends that institutions should not only be more efficient, but should also be 
more responsive to the needs of their students and accountable to the taxpayer. To come back to 
Chris Ball (1985), he eventually conceded that “quality in education” is a subject extraordinarily 
difficult to come to grips with, and full of pitfalls. 

The above analysis of quality is pertinent to our conception of assessment and how we go about 
assuring the quality of our students’ written work. It is our view that the challenges of assessment 
are more pronounced in ODL institutions than in campus-based, full-time contact institutions. 
Elsewhere we have argued that 

“the concept of open and distance learning suggests an educational approach designed to reach 
learners in their homes/offices/shops etc, provide learning resources for them to qualify without attending 
formal classes in person, or create opportunities for lifelong learning, no matter where or when they want 
to study” (Letseka & Pitsoe, 2012, p. 222). 

It seems such a challenge to sufficiently ascertain what students really know, what they can do, 
and what attitudes they might have acquired as a result of interacting with the learning material in 
an ODL environment. The reason for this is not hard to find. Even in a full-time contact education 
institution it is not easy to ascertain whether the students actually interact with the learning material 
in the first place, or whether they make time to interact with the learning material at all. The 
challenge becomes more profound in the learning environment where the students are not required 
to attend “formal classes in person.” 

We may, on the surface be misconstrued as ODL sceptics. Our scepticism, if there is any, is borne 
out of the practical challenges at UNISA where students submit well-written, well-argued and coher¬ 
ent assignments, but perform dismally in the examination. On numerous occasions we have been 
left wondering whether a relative or a close friend, who has probably graduated from the programme, 
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could have assisted one of our students in writing the assignments given that during the examina¬ 
tion some of the students perform poorly in the conceptual, philosophic and contemplative questions. 
Given the very nature of the ODL environment, and regardless of having watertight assessment 
rubrics and criteria, there is evidence of conspicuous inconsistencies in performance among our 
students between assignments and the examination. There are other challenges too, especially with 
respect to the majority of students who live in the rural areas and previously disadvantaged parts 
of South Africa where basic services such as the postal, electricity, Internet and online connectivity 
continue to be elusive. If there should be any doubt at all about students’ interaction with their learn¬ 
ing material, our view is that there is no way of knowing whether we are indeed assessing what we 
should be assessing in the first place. We do not regard ourselves as prophets of ODL doom. And 
while we recognise that there are pockets of excellence among some of our ODL students, we also 
acknowledge that there are pockets of our ODL students whose responses to assignment and 
examination questions leave much to be desired and suggest that their attitudes might not be 
changing as a result of enrolling in ODL programmes. On the positive side though, and consistent 
with the above exposition of the principles of quality and assessment, we are systematically gather¬ 
ing, analysing and interpreting the evidence; and using the resulting information to document, 
explain, and improve the performance of our students. 

Conclusion 

In this article we have sketched various conceptions of assessment. We have acknowledged that 
there is general consensus among advocates of assessment that the purpose of assessment should 
be mainly to improve the quality of teaching and learning. And given that teaching and learning 
involve both lecturers and students, we suggested that assessment should be a collaborative 
endeavour between lecturers and students in order to produce the best performance in teaching 
and learning. We argued that assessment should involve making lecturers and students’ expecta¬ 
tions explicit and public; it should lead to a common understanding on appropriate criteria and 
standards by which quality learning will be assessed; it should lead to gathering, analysis and 
interpretation of evidence to determine how well performance matches expectations and standards. 
In this regard assessors should draw on the information to document, explain, and improve teach¬ 
ing and learning performance. We raised concerns about the challenges of assessment in the ODL 
sector. We argued that while ODL is ideally suited for breaking the barriers of distance by providing 
access to higher education to students without the actual physical contact with their lecturers, there 
is the unintended consequence that quality assessment might be compromised. We argued often 
that lecturers have no way of knowing whether the assignments that the students submit are their 
own authored work or a product of assisted collaboration. This suspicion gains credence especially 
when students perform poorly in the final examination, which is their summative assessment. 

Note 

1 The notion of “Education as initiation” was made prominent by British philosopher of education Richard 
Stanley Peters in his inaugural lecture which he delivered at the Institute of Education, University of 
London, 9 December 1963. 
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